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Abstract 

The multidimensional assignment problem (MAP) (abbreviated s-AP in the case of s 
dimensions) is an extension of the well-known assignment problem. The most studied 
case of MAP is 3-AP, though the problems with larger values of s have also a number 
of applications. In this paper we consider four fast construction heuristics for MAP. 
One of the heuristics is new. A modification of the heuristics is proposed to optimize 
the access to slow computer memory. The results of computational experiments for 
several instance families are provided and discussed. 



1 Introduction 

The Multidimensional Assignment Problem (MAP) (abbreviated s-AP in the 
case of s dimensions) is a well-known optimization problem with a host of ap- 
plications (see, e.g., [H [6l [7] for 'classic' applications and [H [17] for recent 
applications in solving systems of polynomial equations and centralized multi- 
sensor multitarget tracking). In fact, several applications described in [5|, f6l fl7] 
naturally require the use of s-AP for values of s larger than 3. 

MAP is an extension of a well-known Assignment Problem (AP) which is 
exactly two dimensional case of MAP. While AP can be solved in a polynomial 
time [II], s-AP for s > 2 is NP-hard [S]. 

For a fixed s > 2, the s-AP is stated as follows. Let Xi — X2 = . . . = 
Xg = {1,2, . . . ,n}. We will consider only vectors that belong to the Cartesian 
product X = Xi X X2 X ... x Xg- Each vector e € X is assigned a non- 
negative weight w{e). For a vector e € X, the component ej denotes its jth 
coordinate, i.e., ej € Xj. A collection oit < n vectors e^, e^, . . . , e* is a (feasible) 
partial assignment if e*j ^ j holds for each i ^ k and j G {1, 2, . . . , s}. The 
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weight of a partial assignment A is w{A) = J2i=i w{e'^). An assignment (or /mZ^ 
assignment) is a partial assignment with n vectors. The objective is to find an 
assignment of minimum weight. 

The 3-AP is the most studied case of MAP so far. Aiex et al. introduce a 
Greedy Randomized Adaptive Search Procedure for 3-AP in [T]; an exact algo- 
rithm for 3-AP is proposed by Balas and Saltzman in [?j ; Grama and Spieksma 
discuss some special cases of 3-AP and propose approximation algorithms for 
them. A memetic approach is tried by Huang and Lim in [11] . The more general 
case of s-AP for arbitrary values of s is less studied. The most recent research 
by Gutin, Goldengorin and Huang overviews the previous results and discusses 
the worst case analysis of several MAP construction heuristics [TD]. 

2 Heuristics 

There are three construction heuristics for MAP known from the literature: 
Greedy, Max- Regret [Hin|, and ROM [TU]. In this paper, we propose a mod- 
ification of ROM, Shift-ROM, and compare all four heuristics with respect to 
solution quality and running time. 

2.1 Greedy heuristic 

The Greedy heuristic starts with an empty partial assignment A = On each of 
n iterations Greedy finds a vector e G X of minimum weight, such that A U {e} 
is a feasible partial assignment, and adds it to ^. 

The time complexity of Greedy heuristic is 0(n^ -t- (n — 1)'' + . . . + 2* + 1) = 
0{n^~^^) (if the Greedy algorithm is implemented via sorting of all the vectors 
according to their weights, the algorithm complexity is 0(n^ ■ logn'') however 
this implementation is inefficient, see Subsection l3.1[) . 

2.2 IVIax-Regret 

The Max- Regret heuristic was first introduced in [4J for 3-AP and its modifica- 
tions for s-AP were considered in [5]. 

Max-Regret proceeds as follows. Initialize partial assignment A = 0. Set 
Vd = {1, 2, . . . ,n} for each 1 < d < s. For each dimension d and each coordinate 
value V ^ Vd consider every vector e G X' such that = u, where X' C X 
is the set of 'available' vectors, i.e., A U {e} is a feasible partial assignment 
if and only if e G X' . Find two vectors e-'^min and e^min in the considered 
subset Yd^v — {g ^ X' : ed — v} such that e^min = argmiUggy-^ ^ ?x;(e), and 
e^min — argmiUggy^ . J w(e). Select the pair {d,v) that corresponds to 

the maximum difference u;(e^min) — w(e^min) and add the vector e^min for the 
selected (d, v) to A. 

The time complexity of Max-Regret is 0(s • + s • (n — 1)'' + ... + s • 2" + s) = 
0(s-n"+i). 
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2.3 ROM 

The Recursive Opt Matching (ROM) is introduced in [TU] as a heuristic of large 
domination number (see [10' for definitions and results in domination analysis). 
ROM proceeds as follows. Initialize the assignment A with the trivial vectors: 
A'' — {i,i, . . . ,i). On each jth iteration of the heuristic, j — 1, 2, . . . , s — 1, 
calculate an n x n matrix Mi^y = J2e&Y{j i v)'^i^)j where Y{j,i,v) is a set of 
all vectors e ^ X such that the first j coordinates of the vector e are equal to 
the first j coordinates of the vector A^ and the {j + l)th coordinate of e is v. 
Y{j,i,v) = {e £ X : Ck = A\, 1 < k < j and ej+i = v}. Let permutation tt be 
a solution of the 2-AP for the matrix M. Set = 7r(i) for each 1 < i < n. 

The time complexity of ROM heuristic is 0{{n^ + n^) + (n"^^ + n"^) + . . . + 
(n2 +n3)) ©(n" + sn3). 

2.4 Shift-ROM 

A disadvantage of the ROM heuristic is that it is not symmetric with respect to 
the dimensions. For example, if the vector weights do not depend significantly 
on the last coordinate then the algorithm is likely to work badly. Shift-ROM is 
intended to solve this problem by trying ROM for different permutations of the 
instance dimensions. However, we do not wish to try all s! possible dimension 
permutations as that would increase the running time of the algorithm quite 
significantly and instead we use only s different permutations: {X1X2 ■ ■ ■ Xg), 
{XsXiX2 ■ ■ ■ Xs^i), {Xs-iXsXiX2 ■ ■ ■ Xs-2), ■ ■ ■ , {X2X3 . . . XgXi). 

In other words, on each run Shift-ROM applies ROM to the problem; upon 
completion, it renumbers the dimensions for the next run in the following way: 
Xi := X2, X2 := ^3, Xs-i := Xs, Xs := Xi. After s runs, the best 

solution is selected. 

The time complexity of Shift-ROM heuristic is ©((n* + sn'^) • s) — 0{sn^ + 
s^n^). 

2.5 Time Complexity Comparison 

Now we can gather the information about the time complexity of the consid- 
ered heuristics. The following table shows the time complexity of each of the 
heuristics for different values of s. 

Greedy Max-Regret ROM Shift-ROM 

Arbitrary s 0{n'+^) 0{sn''+^) Oin" + sn^) Oisn' + s'^n^) 
Fixeds = 3 0{n^) 0{n'^) 0{n^) 0{n^) 

Fixed s > 4 0(71^+1) 0(n"+i) 0{n'') 0(n") 

3 Performance Notes 

Modern computer architecture is complex and, hence, not every operation takes 
the same time to execute. In a standard computer model it is assumed that all 
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the operations take approximately the same time. We will use a more sophisti- 
cated model in our further discussion. The idea is to differentiate fast and low 
memory access operations. 

The weight matrix of a MAP instance is normally stored in the Random 
Access Memory (RAM) of the computer. RAM's capacity is large enough for 
the very large instances, e.g., nowadays RAM of a common desktop PC is able 
to hold the weight matrix for 3-AP with n = 700, i.e., 3.43 • 10^ weight^ RAM 
is a fast storage; one can load gigabytes of data from RAM in one second. 
However, RAM has a comparatively high latency, i.e., it takes a lot of time for 
the processor to access even a small portion of data in RAM. The processor 
cache is intended to minimize the time spent by the processor for waiting for 
RAM response. 

The processor cache exploits two heuristics; firstly, if some data was recently 
used then there is a high probability it will be used again soon and, secondly, 
the data is used successively, i.e., if some portion of data is used now then it is 
probably that the successive portion of data will be used soon. As an example, 
consider an in place vector multiplication algorithm: on every iteration the 
algorithm loads a value from the memory, multiplies it and saves the result at 
the same memory position. So, the algorithm accesses every portion of data 
twice and the data is accessed successively, i.e., the algorithm accesses the first 
element of the data, then it accesses the second element, the third etc. 

Processor cachqj is a temporary data storage, relatively small and fast, usu- 
ally located on the same chip as the processor. It contains several cache lines 
of the same size; each cache line holds a copy of some fragment of the data 
stored in RAM. Each time the processor needs to access some data in RAM it 
checks whether this data is already presented in the cache. If this is the case, 
it accesses this data in the cache instead. Otherwise a 'miss' is detected, the 
processor suspends, some cache line is freed and a new portion of data is loaded 
from RAM to cache. Then the processor resumes and accesses the data in the 
cache as normally. Note that in case of a 'miss' the system loads the whole cache 
line that is currently 64 bytes on the most of the modern computers [7 and this 
size tends to grow with the development of computer architecture. Thus, if a 
program accesses some value in the memory several times in a short period of 
time it is very probably that this data will be loaded from RAM just once and 
then will be stored in the cache so the access time will be minimal. Moreover, if 
some value is accessed and, thus, loaded from RAM to the processor cache, it is 
probably that the next value is also loaded since the cache line is large enough 
to store several values. 

With respect to MAP heuristics, there are two key rules for improving the 
memory subsystem performance: 

1. The successive access to the weight matrix (scan), i.e., access to the matrix 
in the order of its alignment in the memory, is strongly preferred (we use 

^ Here and further we assume that every weight is represented with a 4 byte integer. The 
calculations are provided for 2 Gb of RAM. 

^ We provide a simplified overview of cache, for detailed information, see, e.g., |3]. 
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the row-major order [13] for weight matrix in our implementation of the 
algorithms). Note that if an algorithm accesses, e.g., every second weight 
in the matrix and does it in the right order, the real complexity of this 
scan for the memory subsystem is the same as the complexity of a full 
scan since loading of one value causes loading of several neighbor values. 

2. One should minimize the number of the weight matrix scans as much as 
possible. Even a partial matrix scan is likely to access much more data 
than the processor cache is able to store, i.e., the data will be loaded from 
RAM all over again for every scan. 

Following these rules may significantly improve the running time of the 
heuristics. In our experiments, the benefit of following these rules was a speedup 
of roughly speaking 2 to 5 times. 

3.1 Greedy heuristic optimization 

A common implementation of the greedy heuristic for combinatorial optimiza- 
tion problem involves sorting of all the weights in the problem. In case of MAP 
this approach is inefficient since we actually need only n vectors from the set. 
Another natural implementation of the Greedy heuristic is to scan all available 
vectors and to choose the lightest one on each iteration but it is very unfriendly 
with respect to the memory subsystem: it performs n scans of the weight matrix. 

We propose a combination of these approaches; our algorithm proceeds as 
follows. Let A = be a partial assignment and B an array of vectors. While 
\A\ < n, i.e., A is not a full assignment, the following is repeated. We scan 
the weight matrix to fill array B with k vectors corresponding to k minimal 
weights in non-decreasing order: if the weight of the current vector is less than 
the largest weight in B then we insert the current vector to B in an appropriate 
position and, if necessary, remove the last element of B. Then, for each vector 
e E B, starting from the lightest, we check whether A U {e} is a feasible partial 
assignment and, if so, add e to A. Note, that during the second and further 
cycles we scan not the whole weight matrix but only a subset X' C X the 
vectors that can be included into the partial assignment A with the feasibility 
preservation: A U {x} is a partial assignment for any x G X' . The size of the 
array B is calculated as fc = min{64, |^'|}. The constant 64 is obtained from 
experiments. 

The algorithm is especially efficient on the first iterations, i.e., in the hardest 
part of its work, while the most of the vectors are feasible. However, there 
exists a bad case for this heuristic. Assume that the weight matrix contains 
a lot of vectors of the minimal weight Wmin- Then the array B will be filled 
with vectors of the weight Wmin at the beginning of the scan and, thus, it will 
contain a lot of similar vectors (recall that the weight matrix is stored in the 
row-major order and only the last coordinates are varied at the beginning of the 
scan, so all the vectors processed at the beginning of the scan are likely to have 
the same first coordinates). As a result, selecting the first of these vectors will 
cause infeasibility for the other vectors in B. We use an additional heuristic to 
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decrease the running time of the Greedy algorithm for such instances. Let Wmin 
be the minimum possible weight: Wmin = niiuegx' ^(6) (sometimes this value is 
known like for Random instance family it is 1, see Section 21 or one can assume 
that Wmin = — oo). If it occurs during the matrix scan that all the vectors in 
B have the weight uimin, i-e., w(Bi) = Wmin for every 1 < i < fc, then the rest 
of the scan can be skipped because there is certainly no vector lighter than the 
maximum weight vector in B. Moreover, it is safe to update Wmin with the 
maximum weight of a vector in B every time before the next matrix scan. 

3.2 Max-Regret Heuristic Optimization 

The Max-Regret heuristic naturally requires 0{ii?s) weight matrix partial scans. 
Each of these scans fixes one coordinate and, thus, every available vector e G X' 
(see Subsection 13. ip is accessed s times during each iteration, and this access is 
very inefficient when the last coordinate is fixed (recall that the weight matrix 
is stored in a row-major order and, thus, if the last coordinate is fixed then 
the algorithm accesses every nth value in the memory, i.e., the access is very 
non-successive and one can assume that this scan will load the whole weight 
matrix from RAM to cache). In our more detailed computer model, the time 
complexity of the non-optimized Max-Regret is 0((s — 1) • n*+^ -I- n*"*"^). 

We propose another way to implement Max-Regret. Let us scan the whole 
available vectors set X' on each iteration. Let L be an n x s matrix of the lightest 
vector pairs: L\ ^ and Lf ^ are the lightest vectors when the jth coordinate is 
fixed as i, and w{Lj j) < w{Lfj). To fill the matrix L we do the following: 
for every vector e G X' and for every coordinate 1 < d < s check: if w{e) < 
w^(^L.d)' ^e,M = ^L.d ^e„d = ^- Otherwise if w{e) < Lj^^^, set 

^ — e. Thus, we update the L^^^d item of the matrix with the current e if 
'w{e) is small enough. Having the matrix L, we can easily find the coordinate 
d and the fixed value v such that w{Ll ^) — w{Ll ^) is maximized. The vector 
Ll ^ is added to the solution and the next iteration of the algorithm is executed. 

The proposed algorithm performs just n partial scans of the weight matrix. 
The matrix L is usually small enough to fit in the processor cache, so the access 
to L is fast. Thus, the time complexity of the optimized Max-Regret in our more 
detailed computer model is 0{n''^^). 

3.3 ROM Heuristic Optimization 

The ROM heuristic can be implemented in a very friendly with respect to the 
memory access way. On the first iteration it fixes the first two coordinates (n^ 
combinations) and enumerates all vectors with these fixed coordinates. Thus, it 
scans the whole weight matrix successively. On the next iteration it fixes three 
coordinates (n^ combinations as the second coordinate depends on the first one), 
and enumerates all vectors with these fixed coordinates. Thus, it scans ti^ solid 
j^s-3_gj2e fragments of the weight matrix; further iterations are similar. As a 
result, the time complexity of ROM in our more detailed computer model is the 
same as in a simple one: 0{n^ + srv^). 
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3.4 Shift-ROM Heuristic Optimization 

The Shift-ROM heuristic is an extension of ROM; it simply runs ROM s times, 
starting it from different coordinates. However, not every run of ROM is efficient 
as a part of Shift-ROM. Let us consider the case when the first iteration of ROM 
fixes the last two coordinates. For each of the combinations of the last two 
coordinate values, the heuristic scans the whole weight matrix with the step 

between the accessed weights, i.e., the distance between the successively 
accessed weights in the memory is elements, which is very inefficient. A 
similar situation occurs when the first and the last dimensions are fixed. 

To avoid this disadvantage, we propose the following algorithm. Let M'^ be 
an n X n matrix for every 1 < d < s. Initialize Mfj = for every 1 < d < s 
and 1 < i, j < n. For each vector e d X and for each 1 < d < s set M'^ „ = 
Mg^ e^+i + w{e) (here we assume that Cg+i — ei). Now the matrices M'^ can be 
used for the first iteration of every ROM run. 

When applying this technique, only one full matrix scan is needed for the 
heuristic and this scan is successive. There are several other inefficient itera- 
tions like fixing of the last three coordinates but they influence the performance 
insigniflcantly. 

4 Test Bed 

In this paper we consider four instance families. 

Random instance family is a family of random instances, i.e., w{e) is chosen 
arbitrary for each e € X. Each weight is a uniformly distributed integer number 
in the interval [1, 100]. This instance family is used in [UUKTB] and some other 
papers. 

Composite instance family is a family of semi-random instances. They were 
introduced by Crama and Spieksma for 3-AP as the T problem 7J. We extend 
this family for s-AP case. 

In [7] the 3-AP problem is interpreted as follows. Given a complete tripartite 
graph K = {X1UX2UX3, {Xi x ATa) U (Xi x A3) U (A2 x X3)) , find a subset 
A oi n triangles, A C Xi x X2 x A3, such that every element of Ai U A2 U X3 
occurs in exactly one triangle of A, and the total weight of all the edges covered 
by triangles A is minimized. The weight of a triangle is calculated as the sum 
of weights of its edges; the weight of an edge S Ai x A2 is dj^, the weight 
of an edge (z, j) e A2 x A3 is d^ j, and the weight of an edge e Ai x A3 is 
df j, where d^, d^ , and o?'^ are random n x n matrices of non- negative numbers. 
In our interpretation of the problem, z«(ji,i2,«3) = dl_^ + df^ -I- d^_^ 

We introduce an extension of the T problem from [7] . Let us consider a graph 
G(AiUA2U...UA„ (Ai X A2)U(A2 X A3)U...U(A,_i X A,)U(Ai X A,)), 
where the weight of an edge {i,j) G Ai x A2 is djj, the weight of an edge 
G A2 X A3 is df j, . . . , and the weight of an edge G X^-i x Ag is d^~^, 
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the weight of an edge (z, j) E X^xXg is dfj and , d? , . . . , d" are random nxn 
matrices of non-negative numbers distributed uniformly in the interval [1, 100]. 
The objective is to find a set of n vertex-disjoint s-cycles C C XiX X2 x . . . x 
such that the total weight of all edges covered by the cycles C is minimized. In 
our interpretation of the problem, ^(e) = d\^^g^ + d1^^^^ + . . . + dl~\^^^ + d^^^^^ . 

Additional conditions are applied in [7| to the random matrices d^, d^, and 
d^ , but we do not use these restrictions. 

CS instance set is the instance set used by Crama and Spieksma in 7J for 
the TA problem that is a special case of T, i.e., CS is a subset of the Composite 
instance family. There are three types of instances, 6 instances per each type: 
3 instances of size 33 and 3 instances of size 66. All the instances are of 3-AP. 
The CS instances meet the triangle inequality, i.e., d'(i,j) < d\i,k) + d\k,j) 
for every I € {1, 2, 3} and every fee {1,2,..., n}. For detailed information, 
see [7]. 

GP instance family contains pseudo-random instances with the predefined 
solutions. Predefined instances are generated by an algorithm described by 
Grundel and Pardalos in [9]. The generator is naturally designed for s-AP for 
arbitrary large values of s and n. The GP generator is relatively slow and, thus, 
it was impossible to experiment with large GP instances. 

All the instances for this paper are generated with the standard Miscrosoft 
.NET random generator [15] which is based on the Donald E. Knuth's sub- 
tractive random number generator algorithm |12j . For the seed of the random 
number sequence we use the following number: seed = s + n + i, where i 
is the index of the instance of this type, i G {1, 2, . . . , 10}. The GP gener- 
ator is implemented in C-I--I- programming language and, thus, the standard 
Visual C-l— I- random number generator is used instead; the seed for it is calcu- 
lated in the same way. The generator for GP instances is available on the web 
(http://www.ici.ro/camo/forum/grudel/map.txt). The CS instances and 
solutions are taken from ,http : / / www . econ . kuleuven . ac . be/public/NDBAE03/instaiicesEJ0R . htm 

5 Experimental Results 

We have conducted a number of experiments for the optimized versions of the 
Greedy, Max- Reg ret, ROM, and Shift-ROM heuristic (see Section |3l). The test 
bed is discussed in Section IH 

Every experiment, except the experiments with CS instances, includes 10 
runs for each of the heuristics; so, 10 instances are produced for every ex- 
periment. The evaluation platform is based on AMD Athlon 64 X2 3.0 GHz 
processor. 

The headers in the tables below are as follows: 
s is the number of dimensions of the instance. 
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n is the linear size of the instance, i.e., n — \Xi\ = \X2\ = . . . = \Xs\. 

Best is the average for the best known objective values for the corresponding 
instances. 

Opt. is the best objective value of the instance. This header is applicable to CS 
instances only. 

Solution error, % is the average value, in percent, over the optimal solution: 
error — (value — opt) /opt ■ 100%, where value is the objective value ob- 
tained by the heuristic and opt is the optimal objective value. For the 
instance families where the optimal objective value is unknown, the Best 
value (see above) is used instead. 

Running time, ms is the average running time, in milliseconds. 

Gr is for Greedy. 

M-R is for Max-Regret. 

R is for ROM. 

S-R is for Shift-ROM. 

The results of the experiments with the Random instance family are pre- 
sented in Table [TJ One can see that the solution quality of all the construction 
heuristics is very poor; the error exceeds 200% over the optimum value on av- 
erage for every heuristic. (Note that the best values reported for the Random 
instances are equal or very close to the minimum possible objective values, i.e., 
to n, and, thus, are equal or very close to the optimal objective values; recall 
that the minimum weight of every vector in Random instance family is 1.) 

Shift- ROM outperforms other heuristics with respect to solution quality on 
average. For some instances Max-Regret performs better but this is at the cost 
of much larger running times. Greedy is approximately 100 times faster than 
Shift-ROM and 2000 times faster than Max-Regret (one can assume that the 
speedup heuristic in the Greedy implementation works well in this case since 
Random instances have a lot of vectors of the minimum possible weight) but it 
is not much worse than the other heuristics with respect to solution quality. 

The results of experiments with the Composite instance family are presented 
in Table [H The solution quality here is much better than in the previous 
experiments. Max-Regret produces the best solutions for 3-AP while Shift-ROM 
is the best for s-AP for s > 4. Both Shift-ROM and especially Max-Regret are 
slow; the fastest heuristic is ROM and it produces relatively good solutions. 
Greedy is slower and produces worse solutions that the ROM heuristic for the 
Composite instances. 

Table [3] contains the experimental results for the CS instance set. Note that 
CS contains instances of three types; the instances of the same type are grouped 
together in the table. One can see that the quality of the Shift-ROM heuristic 
is almost always better than the quality of all other heuristics. ROM solution 
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quality is close to Shift- ROM solution quality and it outperforms Max- Regret 
and Greedy with respect to both solution quality and running time. 

For the GP instance family (Table Max-Regret and Shift-ROM show the 
best solution quality; the average error for both heuristics is about 10%. How- 
ever, Shift-ROM maximum error never exceeds 16.1% while Max-Regret error 
reaches up to 25.7%, and Max-Regret is approximately 10 times slower than 
Shift- ROM. ROM is the fastest heuristic for GP and it produces only 1.5 times 
worse solutions than Max- Regret and Shift- ROM. 

6 Conclusion 

The comparison of the construction heuristics considered in this paper shows 
that the selection of a particular heuristic depends on the instance set and 
the quality/time requirements. Greedy is a very fast heuristic for the Random 
instance family; ROM and Shift-ROM perform well for the Composite instances 
due to their dimensionwise nature. However, in most of the cases Max-Regret 
and Shift-ROM, in particular, produce the best solutions. Moreover, Shift-ROM 
is more stable than Max- Regret with respect to both solution quality and running 
time. The ROM heuristic operates significantly faster than Shift-ROM at the 
price of relatively small solution quality decrease. The Greedy heuristic is fast 
but usually produces the worst solutions. 
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Tab. 1; Heuristics comparison for the Random instance family. Every experi- 
ment includes 10 runs. 









bolution 


error, % 






Running 


times, 


ms 


Inst. 


Best 


Gr 


M-R 


R 


S-R 


Gr 


M-R 


R 


S-R 


3rl00 


100.0 


101.0 


18.7 


67.3 


58.2 


7 




814 


9 


46 


3rl50 


150.0 


54.5 


28.8 


33.6 


30.1 


14 


4 


260 


26 


147 


3r200 


200.0 


42.0 


16.8 


15.5 


13.8 


25 


13 


070 


64 


367 


3r250 


250.0 


37.4 


21.4 


8.4 


6.4 


36 


32 


043 


128 


714 


3r300 


300.0 


27.2 


13.4 


3.8 


3.3 


39 


66 


719 


184 


1 229 


3r350 


350.0 


25.0 


16.8 


2.3 


1.8 


49 


113 


559 


292 


1 889 


3r400 


400.0 


22.2 


7.9 


1.5 


0.9 


55 


192 


683 


417 


2 827 


3r450 


450.0 


20.9 


8.7 


0.6 


0.3 


68 


309 


674 


635 


4 115 


4r20 


20.8 


208.7 


185.6 


310.6 


261.5 


1^ 




29 


1 


11 


4r30 


30.0 


206.3 


193.3 


229.7 


203.3 


3 




204 


6 


48 


4r40 


40.0 


188.8 


118.8 


199.8 


158.5 


4 




825 


18 


140 


4r50 


50.0 


105.2 


93.0 


128.2 


118.4 


7 


2 


650 


51 


349 


4r60 


60.0 


107.7 


98.0 


115.2 


100.0 


8 


6 


355 


94 


713 


4r70 


70.0 


85.6 


68.7 


91.4 


85.6 


9 


13 


176 


151 


1 357 


4r80 


80.0 


74.0 


48.8 


76.4 


71.1 


12 


27 


285 


278 


2 214 


4r90 


90.0 


38.3 


60.1 


70.2 


57.7 


12 


45 


485 


390 


3 746 


5rl0 


10.5 


545.7 


459.0 


641.9 


363.8 


1 




12 


1 


8 


5rl5 


15.0 


359.3 


338.0 


405.3 


333.3 


2 




130 


6 


59 


5r20 


20.0 


232.0 


221.5 


332.0 


235.5 


2 




663 


23 


237 


5r25 


25.0 


229.2 


156.8 


249.2 


219.2 


3 


2 


448 


69 


719 


5r30 


30.0 


228.7 


220.7 


230.0 


185.3 


4 


7 


057 


161 


1 718 


5r35 


35.0 


155.1 


137.1 


194.6 


165.7 


5 


16 


552 


337 


3 707 


6r6 


6.5 


OOO.O 


QO'i 1 


Q1 9 




n 
U 




5 


n 
U 


r 



6r9 


9.0 


674.4 


548.9 


558.9 


387.8 


1 




63 


4 


49 


6rl2 


12.0 


372.5 


392.5 


440.8 


315.8 


2 




417 


22 


292 


6rl5 


15.0 


336.7 


388.0 


408.0 


313.3 


2 


1 


599 


83 


1 037 


6rl8 


18.0 


321.1 


271.1 


372.8 


282.8 


3 


5 


778 


211 


3 003 


7r4 


4.3 


1134.9 


1337.2 


651.2 


393.0 







2 





2 


7r6 


6.0 


788.3 


768.3 


851.7 


446.7 


1 




28 


2 


33 


7r8 


8.0 


726.3 


761.3 


626.3 


403.8 


1 




218 


16 


239 


7rl0 


10.0 


722.0 


663.0 


541.0 


355.0 


1 


1 


176 


78 


986 


7rl2 


12.0 


420.8 


479.2 


475.0 


345.8 


2 


4 


788 


261 


3 757 


8r4 


4.0 


927.5 


1185.0 


1162.5 


510.0 







6 


1 


9 


8r6 


6.0 


615.0 


688.3 


790.0 


473.3 


1 




184 


14 


218 


8r8 


8.0 


548.8 


477.5 


723.8 


433.8 


1 


1 


987 


145 


2 097 


All avg. 


329.2 


326.1 


340.6 


222.5 


11 


24 913 


119 


1 088 


3-AP 


avg. 


41.3 


16.6 


16.6 


14.3 


37 


91 603 


219 


1 417 


4-AP 


avg. 


126.8 


108.3 


152.7 


132.0 


7 


12 001 


124 


1 072 


5-AP 


avg. 


291.7 


255.5 


342.2 


250.5 


3 


4 477 


99 


1 075 


6-AP 


avg. 


508.6 


504.7 


538.6 


350.4 


2 


1 572 


64 


877 


7-AP 


avg. 


758.5 


801.8 


629.0 


388.9 


1 


1 242 


71 


1 003 


8-AP 


avg. 


697.1 


783.6 


892.1 


472.4 


1 




726 


53 


775 
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Tab. 2: Heuristics comparison for the Composite instance family. Every exper- 
iment includes 10 runs. 



Solution error, % Running times, ms 



Inst. 


Best 


Gr 


M-R 


R 


S-R 


Gr 


M-R 


R 


S-R 


3cl00 


1396.8 


43.4 


27.2 


40.0 


33.8 


15 




870 


9 


60 


3cl50 


1760.2 


38.8 


22.3 


33.4 


30.1 


62 


4 


391 


27 


162 


3c200 


2017.8 


37.2 


19.1 


35.6 


33.5 


158 


12 


979 


68 


401 


3c250 


2276.1 


30.4 


17.0 


35.1 


33.6 


292 


31 


738 


129 


749 


3c300 


2551.4 


27.5 


13.2 


34.7 


32.6 


557 


69 


101 


211 


1 229 


3c350 


2696.4 


30.4 


13.1 


37.9 


36.8 


916 


125 


919 


314 


2 023 


3c400 


3008.5 


27.0 


9J_ 


34.6 


33.2 


1 424 


211 


784 


480 


2 925 


3c450 


3222.1 


24.6 


9.2 


35.1 


33.4 


2 017 


338 


248 


685 


4 282 


4c20 


875.7 


40.8 


32.7 


24.8 


16.7 


2 




34 


1 


10 


4c30 


930.1 


51.3 


38.0 


27.0 


21.2 


10 




222 


6 


48 


4c40 


1040.0 


50.3 


41.2 


32.3 


27.2 


30 




919 


19 


145 


4c50 


1139.1 


58.7 


40.4 


38.1 


30.3 


83 


2 


700 


47 


356 


4c60 


1251.0 


53.0 


35.6 


32.8 


27.0 


154 


6 


760 


93 


721 


4c70 


1360.7 


48.5 


33.9 


31.5 


27.0 


287 


14 


678 


173 


1 332 


4c80 


1449.5 


47.8 


33.7 


30.8 


27.5 


543 


28 


037 


284 


2 259 


4c90 


1544.9 


45.2 


25.4 


29.3 


25.3 


825 


51 


083 


457 


3 672 


5cl0 


812.6 


38.2 


26.0 


12.1 




2 




14 


1 


8 


5cl5 


923.1 


41.7 


33.4 


19.8 


10.6 


9 




147 


7 


57 


5c20 


988.8 


53.2 


40.2 


21.2 


16.3 


36 




697 


24 


226 


5c25 


1026.0 


54.2 


46.9 


26.6 


19.5 


117 


2 


577 


11 


699 


5c30 


1091.5 


63.0 


47.5 


30.2 


23.7 


276 


7 


168 


168 


1 666 


5c35 


1171.9 


ou.u 


A7 S 
^ / .o 


28 1 




Ooo 


18 


804 


"^81 
Ool 


O DC5U 


6c6 


817.6 


23.5 


21.0 


9.0 


3.5 


1 




5 


1 


5 


6c9 


911.4 


33.2 


28.8 


11.1 


6.1 


7 




66 


4 


52 


6cl2 


1025.9 


38.4 


31.4 


15.1 


9.9 


39 




412 


23 


273 


6cl5 


1011.1 


44.4 


39.4 


19.7 


14.3 


136 


1 


956 


99 


989 


6cl8 


1073.7 


49.5 


44.1 


18.5 


14.8 


387 


6 


657 


246 


2 919 


7c4 


757.4 


16.5 


12.0 


5.2 


1.2 







2 





2 


7c6 


940.9 


29.0 


23.4 


5.6 


2.3 


4 




29 


2 


32 


7c8 


1000.4 


29.3 


26.7 


12.9 


5.1 


25 




248 


11 


240 


7cl0 


1086.1 


36.8 


34.2 


11.0 


5.6 


124 


1 


278 


92 


998 


7cl2 


1159.3 


45.9 


39.5 


15.2 


8.4 


438 


5 


048 


289 


3 703 


8c4 


841.3 


15.9 


12.5 


4.0 


1.0 


1 




6 


1 


9 


8c6 


1074.1 


26.8 


24.0 


4.5 


1.8 


23 




183 




218 


8c8 


1148.3 


34.2 


31.7 


10.6 




203 


2 


039 


129 


2 069 


All avg. 


39.7 


29.2 


23.2 


18.5 


280 


27 051 


131 


1 092 


3-AP 


avg. 


32.4 


16.4 


35.8 


33.4 


680 


99 379 


240 


1 479 


4-AP 


avg. 


49.4 


35.1 


30.8 


25.3 


242 


13 054 


135 


1 068 


5-AP 


avg. 


51.8 


40.3 


23.0 


16.5 


171 


4 901 


109 


1 056 


6-AP 


avg. 


37.8 


32.9 


14.7 


9.7 


114 


1 819 


75 


848 


7-AP 


avg. 


31.5 


27.2 


10.0 


4.5 


118 


1 321 


80 


995 


8-AP 


avg. 


25.6 


22.7 


6.4 


2.4 


76 




743 


48 


765 
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Tab. 3: Heuristics comparison for the CS instance family. 



Solution error, % Running times, ms 



Inst. 


n 


Opt. 


Gr 


M-R 


R 


S-R 


Gr 


M-R 


R 


S-R 


3DA99N1 


33 


1608.0 


24.5 


19.9 


0.6 


M 


0.9 


11.4 


0.4 


1.8 


3DA99N2 


33 


1401.0 


19.3 


10.3 


1.1 


M 


0.8 


11.5 


0.4 


2.0 


3DA99N3 


33 


1604.0 


15.3 


15.3 


0.3 


as 


0.7 


11.0 


0.4 


1.9 


3DA198N1 


66 


2662.0 


23.7 


17.0 


1.1 


0.2 


4.9 


156.0 


2.8 


14.5 


3DA198N2 


66 


2449.0 


33.1 


36.0 


2.0 


09 


5.7 


187.2 


2.8 


13.8 


3DA198N3 


66 


2758.0 


17.4 


26.0 


1.6 


M 


4.9 


156.0 


2.7 


17.6 


3DIJ99N1 


33 


4797.0 


6.6 


4.8 


1.8 


M 


2.4 


11.3 


OA 


2.2 


3DIJ99N2 


33 


5067.0 


5.6 


3.3 


1.9 


1.3 


2.2 


11.2 


0.5 


2.4 


3DIJ99N3 


33 


4287.0 


7.0 


6.2 


1.3 




1.8 


11.3 


04 


2.1 


3DI198N1 


66 


9684.0 


6.1 


4.4 


1.4 


OJ 


15.7 


124.8 


2.8 


15.6 


3DI198N2 


66 


8944.0 


6.9 


4.9 


2.1 


2.1 


17.7 


140.4 


3.0 


17.2 


3DI198N3 


66 


9745.0 


7.0 


6.2 


1.8 


1.2 


16.7 


171.6 


3.0 


15.3 


3D1299N1 


33 


133.0 


6.8 


5.3 


4.5 


L5 


0.6 


10.5 


0.3 


1.7 


3D1299N2 


33 


131.0 


8.4 


3.8 


6.1 


M 


0.6 


10.3 


0.4 


1.8 


3D1299N3 


33 


131.0 


7.6 


M 


3.1 




0.5 


10.3 


0.3 


1.7 


3D1198N1 


66 


286.0 


5.9 




3.1 


3.1 


4.6 


156.0 


2.5 


13.2 


3D1198N2 


66 


286.0 


3.1 


3.5 


3.1 


2.4 


3.9 


156.0 


2.6 


13.5 


3D1198N3 


66 


282.0 


7.4 


Z5 


4.3 


3.9 


4.3 


140.4 


2.3 


13.1 


All avg. 






11.8 


9.7 


2.3 


M 


4.9 


82.6 


1.6 


8.4 
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Tab. 4: Heuristics comparison for the GP instance family. Every experiment 
includes 10 runs. 



Solution error, % Running times, ms 



Inst. 


Opt. 


Gr 


M-R 


R 


S-R 


Gr 


M-R 


R 


S-R 


3gp20 


98.8 


17.2 


18.1 


18.0 


16.1 


0.4 


1.7 


QA 


04 


3gp30 


150.9 


11.0 


11.9 


14.1 


13.8 


1.0 


7.7 


0.3 


1.4 


3gp40 


197.4 


12.4 


10.5 


13.9 


13.6 


2.0 


23.4 


07 


3.1 


3gp50 


251.5 


9.9 


9.6 


12.2 


11.3 


4.0 


59.0 


M 


6.3 


3gp60 


286.1 


8.9 


8.4 


13.1 


12.1 


6.4 


126.7 


1.9 


11.9 


3gp70 


343.0 


8.3 


u. 


12.1 


11.4 


10.6 


215.3 


3.1 


16.9 


3gp80 


403.7 


7.6 


6.7 


11.6 


10.7 


17.5 


368.2 


47 


25.3 


3gp90 


434.5 


7.5 


5.9 


11.4 


10.5 


25.9 


578.8 


6A 


35.8 


SgplOO 


504.4 


5.6 


5.8 


9.8 


9.4 


40.0 


803.4 


8.7 


47.4 


4gplO 


51.5 


22.5 


19.8 


17.7 


12.6 


0.5 


1.4 


M 


0.8 


4gpl5 


69.6 


16.8 


13.9 


9.2 


7.0 


1.6 


7.9 




3.3 


4gp20 


106.1 


11.7 


10.9 


5.3 


4.5 


5.5 


31.5 


1.4 


10.0 


4gpz;c) 


1 '^9 Q 

ioz.y 


S 1 
O.l 


o.o 


O.O 


O. 1 






o. z 


97 


4gp30 


145.2 


8.8 


8.9 


2.2 


1.5 


34.5 


205.9 


6.7 


50.6 


5gp4 


20.1 


30.3 


23.4 


31.8 


15.9 


0.2 


0.3 




03 


5gp6 


26.9 


32.0 


13.8 


21.2 


14.5 


0.4 


0.8 


(12 


08 


5gp8 


36.3 


28.7 


23.1 


17.6 


12.7 


0.9 


3.7 


OJ 


2.8 


5gpl0 


49.6 


17.7 


10.1 


9.5 


7.1 


2.7 


13.2 


0.8 


8.5 


5gpl2 


66.2 


12.8 


8.9 


9.2 


6.3 


6.4 


36.1 


M 


19.1 


6gp4 


20.4 


23.5 


0.0 


28.4 


14.2 


0.3 


0.4 


M 


05 


6gp6 


30.3 


30.4 


8.9 


18.8 


11.9 


1.5 


5.4 


M 


4.5 


6gp8 


41.8 


24.6 


1.4 


13.6 


8.6 


5.2 


33.0 




25.8 


7gp2 


10.9 


47.7 


25.7 


13.8 


5^ 


0.1 


0.1 


00 


02 


7gp3 


13.8 


33.3 


10.1 


34.8 


13.8 


0.2 


0.3 


oo 


0.5 


7gp4 


20.2 


31.7 


0.0 


30.2 


13.4 


0.4 


1.6 


02 


2.2 


7gp5 


25.6 


27.0 


5.9 


19.5 


8.6 


1.4 


8.5 


08 


8.7 


8gp2 


9.9 


36.4 


10.1 


19.2 


7.1 


0.1 


0.1 


OO 


0.3 


8gp3 


16.3 


54.0 


0.0 


23.9 


11.7 


0.3 


0.7 


02 


1.3 


8gp4 


19.2 


21.4 


6.8 


28.1 


8.3 


1.0 


8.2 


08 


8.8 


All avg. 




21.0 


10.2 


16.3 


10.2 


6.4 


91.0 


L6 


11.2 


3-AP avg. 




9.8 


9.4 


12.9 


12.1 


12.0 


242.7 


M 


16.5 


4-AP avg. 




13.6 


12.4 


7.6 


5.8 


11.5 


68.2 


2.4 


18.5 


5-AP avg. 




24.3 


15.9 


17.9 


11.3 


2.1 


10.8 


0^ 


6.3 


6-AP avg. 




26.2 


3.4 


20.3 


11.6 


2.3 


13.0 


1.0 


10.3 


7-AP avg. 




34.9 


10.4 


24.6 


10.3 


0.5 


2.6 


(13 


2.9 


8-AP avg. 




37.2 


5.6 


23.7 


9.0 


0.5 


3.0 


OJ 


3.5 



